Expanding Training Sets with Unlabeled Samples by Learned Attributes

Designing generalizable classifiers for visual categories is an active research area and has led to the development of many sophisticated classifiers in vision and machine learning [9]. Building a good training set with minimal supervision is a core problem in training visual category recognition algorithms [1]. A good training set should span the appearance variability of its category. While the internet provides a nearly boundless set of potentially useful images for training many categories, a challenge is to select the relevant ones – those that help to change the decision boundary of a classifier to be closer to the best achievable. So, given a relatively small initial set of labeled samples from a category, we want to mine a large pool of unlabeled samples to identify visually different examples without human intervention. To expand the boundary of a category to an unseen region, we propose a method that selects unlabeled samples based on their attributes. The selected unlabeled samples are not always instances from the same category, but they can still improve category recognition accuracy, similar to [4, 5]. We use two types of attributes: category-wide attributes and example-specific attributes. The categorywide attributes find samples that share a large number of discriminative attributes with the preponderance of training data. The example-specific attributes find samples that are highly predictive of the hard examples from a category the ones poorly predicted by a leave one out protocol. We demonstrate that our augmented training set can significantly improve the recognition accuracy over a very small initial labeled training set, where the unlabeled samples are selected from a very large unlabeled image pool, e.g., ImageNet. Our contributions are summarized as follows: 1. We show the effectiveness of using attributes learned with auxiliary data to label unlabeled images without annotated attributes. 2. We propose a framework that jointly identifies the unlabeled images and category wide attributes through an optimization that seeks high classification accuracy in both the original feature space and the attribute space. 3. We propose a method to learn example specific attributes with a small sized training set, used with the proposed framework. We then combine the category wide and the example specific attributes to further improve the quality of image selection by diversifying the variations of selected images. Without modeling the sample distribution and human involvement in the loop, we achieve to find and add samples to categories by attributes, which are helpful for recognition. For more detailed description of the approach and more results, please refer to our main conference version titled “Adding Unlabeled Samples to Categories by Learned Attributes.”1